A Lightweight Distributed Order and Duplication Insensitive Algorithm for Approximate Top-k Queries using Order Statistics

1. APPROXIMATE TOP-K Let {e1, e2, . . . , el} be a set of distinct records in a database, with unique IDs {id1, id2, . . . , idl}. Let A1, A2, . . . , Ap be a set of distinct attributes for each record. For every record ei, the attribute Aj is zero or some positive value. We denote the value of the attribute Aj of record ei by Aj(ei). The sum of the attributes of ei is denoted by Ni = ∑ j Aj(ei). We would like to obtain the list of top k records, ordered by Ni. We present a highly configurable, lightweight, distributed algorithm to solve the above problem approximately, based on order statistics.